Conventional multi party videoconferences use a push model for sending video and audio to clients. The conventional approach uses a centralized transcoding Multipoint Control Unit (MCU). The conventional approach implements audio mixing, video layout composition and conference control entirely on the centralized transcoding MCU. Using this approach, the MCU must implement the user experience rules such as generating the layouts seen by each individual user. Using a push model (i.e., the MCU providing the layout to the clients) makes it difficult for the clients to override the MCU. The video layout composition generally includes decoding of each incoming stream, mixing the video layout composition for each of the participating clients, and encoding the mixed outgoing streams. This generally introduces unwanted delays (latency) in the communication between participating clients.
U.S. Pat. No. 7,561,179/EP1683356 (the contents of both of which are incorporated herein by reference) describes a system and method using a non-transcoding MCU or switching MCU, wherein the non-transcoding MCU receives capability information from the different clients participating in a multi-party videoconference. Based on the received capability information, the non-transcoding MCU instructs the different clients to transmit multimedia streams comprising partial frames adjusted to fit into the capabilities of the receiving clients participating in the videoconference. Two main methods of transmitting multimedia streams comprising partial frames are disclosed in the patent. One being multicasting several video streams of different quality, e.g. resolution, size etc, to the non-transcoding MCU, the other being using scalable video coding techniques such as SVC, wherein multiple levels of video quality are embedded within one stream. In both cases, the non-transcoding MCU then has the option of only passing on the partial frames to the clients that the non-transcoding MCU knows the clients can handle, based on the previously received capability information, or the non-transcoding MCU can function as a multi-cast router, passing on all received partial frames to all participating clients.
However, the use of a centralized unit to determine which video streams and/or resolution of video streams a receiving client should receive potentially limits the flexibility of the user experience and user interface of a client.